Opacity Light Fields: Interactive Rendering of Surface Light Fields with View-Dependent Opacity

We present new hardware-accelerated techniques for rendering surface light fields with opacity hulls that allow for interactive visualization of objects that have complex reflectance properties and elaborate geometrical details. The opacity hull is a shape enclosing the object with view-dependent opacity parameterized onto that shape. We call the combination of opacity hulls and surface light fields the opacity light field. Opacity light fields are ideally suited for rendering of the visually complex objects and scenes obtained with 3D photography. We show how to implement opacity light fields in the framework of three surface light field rendering methods: view-dependent texture mapping, unstructured lumigraph rendering, and light field mapping. The modified algorithms can be effectively supported on modern graphics hardware. Our results show that all three implementations are able to achieve interactive or real-time frame rates.Engineering and Applied Science

Leveraging 3D City Models for Rotation Invariant Place-of-Interest Recognition

Given a cell phone image of a building we address the problem of place-of-interest recognition in urban scenarios. Here, we go beyond what has been shown in earlier approaches by exploiting the nowadays often available 3D building information (e.g. from extruded floor plans) and massive street-level image data for database creation. Exploiting vanishing points in query images and thus fully removing 3D rotation from the recognition problem allows then to simplify the feature invariance to a purely homothetic problem, which we show enables more discriminative power in feature descriptors than classical SIFT. We rerank visual word based document queries using a fast stratified homothetic verification that in most cases boosts the correct document to top positions if it was in the short list. Since we exploit 3D building information, the approach finally outputs the camera pose in real world coordinates ready for augmenting the cell phone image with virtual 3D information. The whole system is demonstrated to outperform traditional approaches on city scale experiments for different sources of street-level image data and a challenging set of cell phone image

NeuroAnimator: Fast Neural Network Emulation and Control of Physics-Based Models

Animation through the numerical simulation of physics-based graphics models offers unsurpassed realism, but it can be computationally demanding. Likewise, finding controllers that enable physics-based models to produce desired animations usually entails formidable computational cost. This paper demonstrates the possibility of replacing the numerical simulation and control of model dynamics with a dramatically more efficient alternative. In particular, we propose the NeuroAnimator, a novel approach to creating physically realistic animation that exploits neural networks. NeuroAnimators are automatically trained off-line to emulate physical dynamics through the observation of physics-based models in action. Depending on the model, its neural network emulator can yield physically realistic animation one or two orders of magnitude faster than conventional numerical simulation. Furthermore, by exploiting the network structure of the NeuroAnimator, we introduce a fast algorithm for learning controllers that enables either physics-based models or their neural network emulators to synthesize motions satisfying prescribed animation goals. We demonstrate NeuroAnimators for passive and active (actuated) rigid body, articulated, and deformable physics-based models

Automated Learning of Muscle-Actuated Locomotion Through Control Abstraction

We present a learning technique that automatically synthesizes realistic locomotion for the animation of physics-based models of animals. The method is especially suitable for animals with highly flexible, many-degree-of-freedom bodies and a considerable number of internal muscle actuators, such as snakes and fish. The multilevel learning process first performs repeated locomotion trials in search of actuator control functions that produce efficient locomotion, presuming virtually nothing about the form of these functions. Applying a short-time Fourier analysis, the learning process then abstracts control functions that produce effective locomotion into a compact representation which makes explicit the natural quasi-periodicities and coordination of the muscle actions. The artificial animals can finally put into practice the compact, efficient controllers that they have learned. Their locomotion learning abilities enable them to accomplish higher-level tasks specified by the animator while guided by sensory perception of their virtual world; e.g., locomotion to a visible target. We demonstrate physics-based animation of learned locomotion in dynamic models of land snakes, fishes, and even marine mammals that have trained themselves to perform "SeaWorld" stunts

Artificial Fishes: Autonomous Locomotion, Perception, Behavior, and Learning in a Simulated Physical World

This paper develops artificial life patterned after animals as evolved as those in the superclass Pisces. It demonstrates a virtual marine world inhabited by realistic artificial fishes. Our algorithms emulate not only the appearance, movement, and behavior of individual animals, but also the complex group behaviors evident in many aquatic ecosystems. We model each animal holistically. An artificial fish is an autonomous agent situated in a simulated physical world. The agent has (i) a three-dimensional body with internal muscle actuators and functional fins, which deforms and locomotes in accordance with biomechanic and hydrodynamic principles, (ii) sensors, including eyes that can image the environment, and (iii) a brain with motor, perception, behavior, and learning centers. Artificial fishes exhibit a repertoire of piscine behaviors that rely on their perceptual awareness of their dynamic habitat. Individual and emergent collective behaviors include caudal and pectoral loc..

The Lumigraph

This paper discusses a new method for capturing the complete appearanceof both synthetic and real world objects and scenes, representing this information, and then using this representation to render images of the object from new camera positions. Unlike the shape capture process traditionally used in computer vision and the rendering process traditionally used in computer graphics, our approach does not rely on geometric representations. Instead we sample and reconstruct a 4D function, which we call a Lumigraph. The Lumigraph is a subsetof the complete plenoptic function that describes the flow of light at all positions in all directions. With the Lumigraph, new images of the object canbegeneratedvery quickly, independent of the geometric or illumination complexity of the scene or object. The paper discusses a complete working system including the capture of samples, the construction of the Lumigraph, and the subsequent rendering of images from this new representation